Impact of a structured reporting template on the quality of HRCT radiology reports for interstitial lung disease.

PURPOSE: This QI study compared the completeness of HRCT radiology reports before and after the implementation of a disease-specific structured reporting template for suspected cases of interstitial lung disease (ILD). MATERIALS AND METHODS: A pre-post study of radiology reports for HRCT of the thorax at a multicenter health system was performed. Data was collected in 6-month period intervals before (June 2019-November 2019) and after (January 2021-June 2021) the implementation of a disease-specific template. The use of the template was voluntary. The primary outcome measure was the completeness of HRCT reports graded based on the documentation of ten descriptors. The secondary outcome measure assessed which descriptor(s) improved after the intervention. RESULTS: 521 HRCT reports before and 557 HRCT reports after the intervention were reviewed. Of the 557 reports, 118 reports (21%) were created using the structured reporting template. The mean completeness score of the pre-intervention group was 9.20 (SD = 1.08) and the post-intervention group was 9.36 (SD = 1.03) with a difference of -0.155, 95% CI [-0.2822, -0.0285, p < 0.0001]. Within the post-intervention group, the mean completeness score of the unstructured reports was 9.25 (SD = 1.07) and the template reports was 9.93 (SD = 0.25) with a difference of -0.677, 95% CI [-0.7871, -0.5671, p < 0.0001]. After the intervention, the use of two descriptors improved significantly: presence of honeycombing from 78.3% to 85.1% (p < 0.0039) and technique from 90% to 96.6% (p < 0.0001). DISCUSSION: Shifting to disease-specific structured reporting for HRCT exams of suspected ILD is beneficial, as it improves the completeness of radiology reports. Further research on how to improve the voluntary uptake of a disease-specific template is needed to help increase the acceptance of structured reporting among radiologists.

Recommended Guideline for Uniform Reporting of Neonatal Resuscitation: The Neonatal Utstein Style.

Clinical research on neonatal resuscitation has accelerated over recent decades. However, an important methodologic limitation is that there are no standardized definitions or reporting guidelines for neonatal resuscitation clinical studies. To address this, the International Liaison Committee on Resuscitation Neonatal Life Support Task Force established a working group to develop the first Utstein-style reporting guideline for neonatal resuscitation. The working group modeled this approach on previous Utstein-style guidelines for other populations. This reporting guideline focuses on resuscitation of newborns immediately after birth for respiratory failure, bradycardia, severe bradycardia, or cardiac arrest. We identified 7 relevant domains: setting, patient, antepartum, birth/preresuscitation, resuscitation process, postresuscitation process, and outcomes. Within each domain, relevant data elements were identified as core versus supplemental. Core data elements should be collected and reported for all neonatal resuscitation studies, while supplemental data elements may be collected and reported using standard definitions when possible. The Neonatal Utstein template includes both core and supplemental elements across the 7 domains, and the associated Data Table provides detailed information and reporting standards for each data element. The Neonatal Utstein reporting guideline is anticipated to assist investigators engaged in neonatal resuscitation research by standardizing data definitions. The guideline will facilitate data pooling in meta-analyses, enhancing the strength of neonatal resuscitation treatment recommendations and subsequent guidelines.

Diseño y validación de una rúbrica analítica para evaluar manuscritos científicos / Design and validation of an analytical rubric for evaluating scientific manuscripts

Introducción: La investigación científica tiende a ser presentada mediante reportes escritos, que inicialmente son denominados manuscritos, la revisión de su calidad es importante en el marco del rigor metodológico y científico. Objetivo: Diseñar y validar una rúbrica analítica para evaluar manuscritos científicos. Material y Métodos: Estudio instrumental. Donde se elabora una rúbrica analítica compuesta por 21 aspectos integrados en seis dimensiones y cuatro niveles de desempeño (Excelente, Bien, Regular, Por mejorar). Se utiliza la metodología de jueces expertos (n= 9), fiabilidad mediante el Alfa de Krippendorff (α) y relación prueba criterio externo e información cualitativa para la mejora de los indicadores. Resultados: El juicio de expertos indicó que los 21 aspectos a evaluar pueden ser considerados validos (V de Aiken ≥ 0.70) y fiables (α ≥ 0.70). Además, las sugerencias de los expertos permitieron mejoras cualitativas al instrumento. La aplicación piloto con un grupo de siete evaluadores indicó que la prueba cuenta con poder predictivo, porque se encontró que las puntuaciones de los jueces disminuyen mientras disminuye la calidad del manuscrito seleccionado. Conclusiones: La rúbrica analítica elaborada es un instrumento válido y fiable que puede ser utilizado para la valoración de manuscritos científicos en el ámbito de la educación médica. El proceso metodológico brinda evidencias solidas de su funcionamiento. A pesar de eso, se anima a continuar revisando el instrumento como parte de su proceso de mejora continua.

Introduction: Scientific research tends to be presented through written reports, which are initially called manuscripts. The review of their quality is important in the framework of methodological and scientific rigor. Objective: To design and validate an analytical rubric to evaluate scientific manuscripts. Material and Methods: Instrumental study. An analytical rubric composed of 21 aspects integrated into six dimensions and four performance levels (excellent, good, fair, to be improved) was developed. The methodology of expert judges (n = 9), reliability by means of Krippendorff's alpha (α), external criterion test, and qualitative information for the improvement of the indicators were used. Results: The expert judgment indicated that the 21 aspects to be evaluated can be considered valid (Aiken's V ≥ 0.70) and dependable (α ≥ 0.70). In addition, the experts' suggestions allowed qualitative improvements to the instrument. The pilot application with a group of seven evaluators indicated that the test has predictive power because it was found that the judges' scores decreased as the quality of the selected manuscript decreased. Conclusions: The analytical rubric elaborated is a valid and reliable instrument that can be used for the assessment of scientific manuscripts in the field of medical education. The methodological process provides solid evidence of its performance. Nevertheless, it is encouraged to continue revising the instrument as part of its continuous improvement process.

Nationwide implementation of a multifaceted tailored strategy to improve uptake of standardized structured reporting in pathology: an effect and process evaluation.

BACKGROUND: Implementation strategies are aimed at improving guideline adherence. Both effect and process evaluations are conducted to provide insights into the success or failure of these strategies. In our study, we evaluate the nationwide implementation of standardized structured reporting (SSR) in pathology. METHODS: An interrupted time series analysis was conducted to evaluate the effect of a previously developed implementation strategy, which consisted of various digitally available elements, on SSR in pathology laboratories. A segmented regression analysis was performed to analyze the change in mean SSR percentages directly after the strategy introduction for pathology reporting and specific subcategories. In addition, we analyzed the change in trend in the weekly percentages after strategy introduction, also for subgroups of tumor groups, retrieval methods, and type of laboratory. The change in SSR use after the strategy introduction was determined for all pathology laboratories. We further conducted a process evaluation in which the exposure to the strategy elements was determined. Experiences of the users with all strategy elements and the remaining barriers and potential strategy elements were evaluated through an eSurvey. We also tested whether exposure to a specific element and a combination of elements resulted in a higher uptake of SSR after strategy introduction. RESULTS: There was a significant increase in an average use of SSR after the strategy introduction for reporting of gastrointestinal (p=.018) and urological (p=.003) oncological diagnoses. A significant increase was present for all oncological resections as a group (p=.007). Thirty-three out of 42 pathology laboratories increased SSR use after the strategy introduction. The "Feedback button", an option within the templates for SSR to provide feedback to the provider and one of the elements of the implementation strategy, was most frequently used by the SSR users, and effectiveness results showed that it increased average SSR use after the strategy introduction. Barriers were still present for SSR implementation. CONCLUSIONS: Nationwide SSR implementation improved for specific tumor groups and retrieval methods. The next step will be to further improve the use of SSR and, simultaneously, to further develop potential benefits of high SSR use, focusing on re-using discrete pathology data. In this way, we can facilitate proper treatment decisions in oncology.

Analysis of 567,758 randomized controlled trials published over 30 years reveals trends in phrases used to discuss results that do not reach statistical significance.

The power of language to modify the reader's perception of interpreting biomedical results cannot be underestimated. Misreporting and misinterpretation are pressing problems in randomized controlled trials (RCT) output. This may be partially related to the statistical significance paradigm used in clinical trials centered around a P value below 0.05 cutoff. Strict use of this P value may lead to strategies of clinical researchers to describe their clinical results with P values approaching but not reaching the threshold to be "almost significant." The question is how phrases expressing nonsignificant results have been reported in RCTs over the past 30 years. To this end, we conducted a quantitative analysis of English full texts containing 567,758 RCTs recorded in PubMed between 1990 and 2020 (81.5% of all published RCTs in PubMed). We determined the exact presence of 505 predefined phrases denoting results that approach but do not cross the line of formal statistical significance (P < 0.05). We modeled temporal trends in phrase data with Bayesian linear regression. Evidence for temporal change was obtained through Bayes factor (BF) analysis. In a randomly sampled subset, the associated P values were manually extracted. We identified 61,741 phrases in 49,134 RCTs indicating almost significant results (8.65%; 95% confidence interval (CI): 8.58% to 8.73%). The overall prevalence of these phrases remained stable over time, with the most prevalent phrases being "marginally significant" (in 7,735 RCTs), "all but significant" (7,015), "a nonsignificant trend" (3,442), "failed to reach statistical significance" (2,578), and "a strong trend" (1,700). The strongest evidence for an increased temporal prevalence was found for "a numerical trend," "a positive trend," "an increasing trend," and "nominally significant." In contrast, the phrases "all but significant," "approaches statistical significance," "did not quite reach statistical significance," "difference was apparent," "failed to reach statistical significance," and "not quite significant" decreased over time. In a random sampled subset of 29,000 phrases, the manually identified and corresponding 11,926 P values, 68,1% ranged between 0.05 and 0.15 (CI: 67. to 69.0; median 0.06). Our results show that RCT reports regularly contain specific phrases describing marginally nonsignificant results to report P values close to but above the dominant 0.05 cutoff. The fact that the prevalence of the phrases remained stable over time indicates that this practice of broadly interpreting P values close to a predefined threshold remains prevalent. To enhance responsible and transparent interpretation of RCT results, researchers, clinicians, reviewers, and editors may reduce the focus on formal statistical significance thresholds and stimulate reporting of P values with corresponding effect sizes and CIs and focus on the clinical relevance of the statistical difference found in RCTs.